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- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
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earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)D Responsive to communication(s) filed on . 



2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
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4) M Claim(s) 1-13 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) ^ Claim(s) 11-13 is/are allowed. 

6) 0 Claim(s) 1-6.8 and 10 is/are rejected. 

7) ^ Claim(s) 7 and 9 is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) Q The drawing(s) filed on is/are: a)D accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)\3 approved b)\3 disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
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application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a P r0> <^^^ 1 ' °^ 

a) Q The translation of the foreign language provisional application has been received. D0VPOP(N(q*^ 
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Attachment(s) fECHNOLOGY CENTER 2100 

1) |3 Notice of References Cited (PTO-892) 4) Q Interview Summary (PTO-413) Paper No(s). . 

2) CD Notice of Drattsperson's Patent Drawing Review (PTO-948) 5) Q Notice of Informal Patent Application (PTO-152) 

3) O Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) O Other: 



U.S. Patent and Trademark Office 

PTO-326 (Rev. 04-01) 



Office Action Summary 



Part of Paper No. 3 



Application/Control Number: 09/726,401 
Art Unit: 2175 



Page 2 



DETAILED ACTION 

Claim Rejections - 35 USC § 103 

1. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1, 3-4 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Wactlar 
et al (U.S. Patent No. 5,835,667) in view of Vaithilingam et al (U.S. Patent No. 6,41 1,724.) 

As to claim 1, Wactlar et al teaches a Method for updating multimedia feature 
information in a multimedia retrieval system (see Abstract), comprising the steps of: 

(a) evaluating a retrieval performance using multimedia feature information (see column 
10, lines 23-32); 

(b) detecting change of retrieval environment based on the retrieval performance 
evaluation (see column 1 1, line 66 through column 12, line 5, where "detecting change of 
retrieval environment" is read on "detect big image changes"); and 

(c) updating the reliability by reflecting the retrieval performance evaluation and the 
retrieval environment change (see column 12, lines 47-51.) 

Wactlar et al does not teach using weight of multimedia features and reliability of the 
weight and does not teach updating the weight of the multimedia feature information. 
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Vaithilingam et al teaches using meta-descriptors to represent multimedia information 
(see Abstract), in which he teaches using weight of multimedia features and reliability of the 
weight (see column 6, lines 37-46); and updating the weight of the multimedia feature 
information (see column 9, lines 56-65.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the 
time the invention was made to have modified Wactlar et al to include using weight of 
multimedia features and reliability of the weight; and updating the weight of the multimedia 
feature information. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Wactlar et al by the teaching of Vaithilingam et al , 
because by using weight of multimedia features and reliability of the weight; and updating 
the weight of the multimedia feature information, the system is enabled to set various 
characteristics to a multimedia file (data) and be able to adjust such characteristics in order to 
improve the quality and performance of the multimedia file, and further be able to measure 
the performances of "before" and "after" modifying such characteristics. 

As to claim 3, Wactlar et al as modified teaches wherein the reliability update is 
proportionally influenced by the retrieval performance (see Vaithilingam et al column 9, 
lines 56-60, where "retrieval performance" is read on "human expert's input".) 
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As to claim 4, Wactlar et al as modified teaches wherein the reliability update is 
proportionally influenced by improvement of the retrieval performances (see Vaithilingam et 
al, column 2, lines 13-15.) 



As to claim 8, Wactlar et al as modified teaches wherein a reliability update is 
proportionally influenced by a ratio of the present retrieval performance to the previous 
retrieval performance (see Wactlar et al column 12, lines 45-52. It is inherent in a system 
which updates the performance of the multimedia files with feedback received from users, 
that the actual update value is the difference between the previous performance and the 
present performance, hence, the update is directly proportional (proportionally influenced) by 
a ratio (percentage) of the present retrieval performance to the previous retrieval 
performance.) 

3. Claims 2 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over Vaithilingam 
et al (U.S. Patent No. 6,41 1,724) in view of Aggarwal et al (U.S. Patent No. 6,408,293.) 

As to claim 2, Vaithilingam et al teaches a Method for updating multimedia feature 
information in a multimedia retrieval system using weight of multimedia features and 
reliability of the weight (see Abstract), comprising the steps of: 

(a) retrieving multimedia (see column 2, lines 13-15) using previous weight (see column 
5, lines 22-39); 
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(c) calculating retrieval performance with respect to the results of present retrieval (see 
column 5, lines 40-54, where "performance" is read on "how clearly defined and compact the 
clusters are"); 

(d) updating the reliability of the present weight by reflecting the calculated retrieval 
performance (see column 9, lines 56-65); and 

(e) updating the present weight using the updated reliability (see column 14, lines 64-67.) 
Vaithilingam et al does not teach (b) receiving one or more user feedbacks with respect to 

results of the multimedia retrieval; and (c) calculating retrieval performance using the one or 
more user feedbacks. 

Aggarwal et al teaches an interactive framework for understanding user's perception of 
multimedia data (see Abstract), in which he teaches receiving one or more user feedbacks 
with respect to results of the multimedia retrieval (see column 3, lines 38-56); and calculating 
retrieval performance using the one or more user feedbacks (see column 5, lines 10-15.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the 
time the invention was made to have modified Vaithilingam et al to include receiving one or 
more user feedbacks with respect to results of the multimedia retrieval; and calculating 
retrieval performance using the one or more user feedbacks. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Vaithilingam et al by the teaching of Aggarwal et ah 
because receiving one or more user feedbacks with respect to results of the multimedia 
retrieval; and calculating retrieval performance using the one or more user feedbacks, would 
enable the system to continuously improve the quality and performance of multimedia files 
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by presenting the files to the users, obtaining users' feedback on the files, and adjusting the 
features of the files based on feedback received from users. 



As to claim 10, Vaithilingam et al teaches a multimedia data structure for a multimedia 
retrieval using weight of the multimedia feature and reliability of the multimedia feature (see 
column 2, line 65 through column 3, line 5), comprising: 

a reliability of present weight updated by reflecting retrieval performance (see column 9, 
lines 56-65, where "updating weights" is read on "updating the meta-descriptors"); 

a present weight updated using the updated reliability (see column 14, lines 64-67.) 

Vaithilingam et al does not teach the performance calculated using one or more user's 
feedbacks with respect to a to multimedia retrieval result obtained using previous weight. 

Aggarwal et al teaches an interactive framework for understanding user's perception of 
multimedia data (see Abstract), in which he teaches the performance calculated using one or 
more user's feedbacks with respect to a to multimedia retrieval result obtained using previous 
weight (see column 2, lines 1 1-17, and see column 5, lines 3-15, where it is taught how the 
user's feedback is used to adjust the measures.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the 
time the invention was made to have modified Vaithilingam et al to include the performance 
calculated using one or more user's feedbacks with respect to a to multimedia retrieval result 
obtained using previous weight. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Vaithilingam et al by the teaching of Aggarwal et ah 



Application/Control Number: 09/726,401 Page 7 

Art Unit: 2175 

because having the performance calculated using one or more user's feedbacks with respect 
to a to multimedia retrieval result obtained using previous weight would enable the system 
where users would customize the properties of the multimedia contents based on their input 
and preferences. 



4. Claims 5 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Wactlar et al 
(U.S. Patent No. 5,835,667) in view of Vaithilingam etal (U.S. Patent No. 6,41 1,724) as 
applied to claims 1, 3-4 and 8 above, and further in view of Aggarwal et al (U.S. Patent No. 
6,408,293.) 

As to claim 5, Wactlar et al as modified teaches a reliability update rate (see Wactlar et 
al, column 12, lines 45-52.) 

Wactlar et al as modified does not teach wherein the update rate is proportionally 
influenced by number of feedbacks participated in calculation of s the retrieval performance. 

Aggarwal et al teaches an interactive framework for understanding user's perception of 
multimedia data (see Abstract), in which he teaches wherein the update rate is proportionally 
influenced by number of feedbacks participated in calculation of s the retrieval performance 
(see column 5, lines 3-15, where it is taught how the user's feedback is used to adjust the 
measures.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the 
time the invention was made to have modified Wactlar et al as modified to include the update 
rate being proportionally influenced by number of feedbacks participated in calculation of s 
the retrieval performance. 
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It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Wactlar et al as modified, by the teaching of Aggarwal 
et al because having the update rate being proportionally influenced by number of feedbacks 
participated in calculation of s the retrieval performance, would enable the system where 
users would customize the properties of the multimedia contents based on their input and 
preferences. 

As to claim 6, Wactlar et al as modified teaches the reliability update (see Wactlar et al, 
column 12, lines 45-52.) 

Wactlar et al as modified does not teach wherein the update is proportionally influenced 
by the difference between the present and previous retrieval performances. 

Aggarwal et al teaches an interactive framework for understanding user's perception of 
multimedia data (see Abstract), in which he teaches wherein the update is proportionally 
influenced by the difference between the present and previous retrieval performances (see 
column 2, lines 11-17.) 

Therefore, it would have been obvious to a person having ordinary skill in the art at the 
time the invention was made to have modified Wactlar et al as modified to include the update 
being proportionally influenced by the difference between the present and previous retrieval 
performances. 

It would have been obvious to a person having ordinary skill in the art at the time the 
invention was made to have modified Wactlar et al as modified, by the teaching of Aggarwal 
etal, because having the update being proportionally influenced by the difference between 
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the present and previous retrieval performances, would enable the system where users would 
customize the properties of the multimedia contents based on their input and preferences by 
calculating the difference between the previous feedback and the most recent feedback 
received by the user. 



Allowable Subject Matter 

5. Claims 11-13 are allowed over the prior art made of record. 

6. The following is a statement of reasons for the indication of allowable subject matter: 

The prior art of record, Wactlar et al (U.S. Patent No. 5,835,667), Vaithilingam et al 
(U.S. Patent No. 6,41 1,724), and Aggarwal et al (U.S. Patent No. 6,408,293), do not disclose, 
teach, or suggest the claimed limitations of (in combination with all other features in the 
claim): 

A method for updating weight of multimedia features using reliability of the weight in a 
multimedia retrieval system using weight among multimedia features and weight among 
elements of the multimedia feature, wherein the Is weight is updated based on the following 
way that: 

(a) the more times the previous feature weights are learned with the feedbacks from the 
user, the less the feature weights are influenced by new feedback; 
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(b) the more recent the feedback is, the more the feedback influence to the feature 
weights update; and 

(c) learning rate of the weights among the multimedia features is higher than that of the 
weights among elements of a multimedia feature, as claimed in claim 1 1 . 

Claim 12 is allowed because it is dependent from the allowed independent claim 11. 

The prior art of record, Wactlar et al (U.S. Patent No. 5,835,667), Vaithilingam et al 
(U.S. Patent No. 6,41 1,724), and Aggarwal et al (U.S. Patent No. 6,408,293), do not disclose, 
teach, or suggest the claimed limitations of (in combination with all other features in the 
claim): 

A multimedia data structure for retrieval multimedia objects using weight among 
multimedia features and weight among elements of the multimedia feature, wherein the 
weight is updated based on the following way that: 

(a) the more times the previous feature weights are learned with the to feedbacks from the 
user, the less the feature weights are influenced by new feedback; 

(b) the more recent the feedback is, the more the feedback influence to the feature 
weights update; and 

(c) learning rate of the weights among the multimedia features is higher is than that of the 
weights among elements of a multimedia feature, in relation to the reliability formula, 
[Reliability x 01d_W + Cur_W] / [Reliability + 1] wherein, 0 < a < 1, and exponential term 
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"a" in the formula for weights of features is less than exponential term "a" in the formula for 
weights of elements of a feature, as claimed in claim 13. 

7. Claims 7 and 9 are objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim 
and any intervening claims. 

8. The following is a statement of reasons for the indication of allowable subject matter: 

The prior art of record, Wactlar et al (U.S. Patent No. 5,835,667), Vaithilingam et al 
(U.S. Patent No. 6,41 1,724), and Aggarwal et al (U.S. Patent No. 6,408,293), do not disclose, 
teach, or suggest the claimed limitations of (in combination with all other features in the 
claim): 

wherein the reliability is calculated by a following formula: 
previous reliability x (1 + reliability increment) + a 

wherein, 

reliability increment: a function that multiplies the difference between the present and 
previous retrieval performance with the number of feedbacks. 

a : constant for making the reliability value proportional to the number of feedbacks in 

same condition, as claimed in claim 7. 
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The prior art of record, Wactlar et al (U.S. Patent No. 5,835,667), Vaithilingam et al 
(U.S. Patent No. 6,41 1,724), and Aggarwal et al (U.S. Patent No. 6,408,293), do not disclose, 
teach, or suggest the claimed limitations of (in combination with all other features in the 
claim): 

wherein the reliability is calculated by a following formula: 
previous reliability x (1 + reliability increment) + a 

wherein, 

reliability increment: a function that multiplies the rate of the present retrieval 
performance to the previous retrieval performance with the number of feedbacks. 

a : constant for making the reliability value proportional to the number of s feedbacks in 

same condition, as claimed in claim 9. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

The following patents are cited to further show the state of art with respect to methods 
and systems of searching and retrieving multimedia data, obtaining user feedbacks on 
multimedia data, and enhancing/updating the data with the obtained users' feedback in 
general: 

U.S. Patent No. 6,208,804 to Ottesen et al . 
U.S. Patent No. 6,347,3 13 to Ma et al . 
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10. Any inquiries concerning this communication or earlier communications from the examiner 
should be directed to Tony Mahmoudi whose telephone number is (703) 305-4887. The 
examiner can normally be reached on Mondays-Fridays from 08:00 am to 04:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dov Popovici, can be reached at (703) 305-3830. 
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